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Abstract. In this paper we develop a geometric approach to convex subdifferential calculus in 
finite dimensions with employing some ideas of modern variational analysis. This approach allows 
us to obtain natural and rather easy proofs of basic results of convex subdifferential calculus in full 
generality and also derive new results of convex analysis concerning optimal value/marginal func¬ 
tions, normals to inverse images of sets under set-valued mappings, calculus rules for coderivatives 
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1 Introduction 

The notion of subdifferential (collection of subgradients) for nondifferentiable convex func¬ 
tions was independently introduced and developed by Moreau [15] and Rockafellar [19] who 
were both influenced by Fenchel [6]. Since then this notion has become one of the most 
central concepts of convex analysis and its various applications, including first of all convex 
optimization. The underlying difference between the standard derivative of a differentiable 
function and the subdifferential of a convex function at a given point is that the subdif¬ 
ferential is a set (of subgradients) which reduces to a singleton (gradient) if the function 
is differentiable. Due to the set-valuedness of the subdifferential, deriving calculus rules 
for it is a significantly more involved task in comparison with classical differential calculus. 
Needless to say that subdifferential calculus for convex functions is at the same level of 
importance as classical differential calculus, and it is difficult to imagine any usefulness of 
subgradients unless reasonable calculus rules are available. 

The first and the most important result of convex subdifferential calculus is the subdif¬ 
ferential sum rule, which was obtained at the very beginning of convex analysis and has 
been since known as the Moreau-Rockafellar theorem. The reader can find this theorem 
and other results of convex subdifferential calculus in finite-dimensional spaces in the now 
classical monograph by Rockafellar [20]. More results in this direction in finite and infinite 
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dimensions with various applications to convex optimization, optimal control, numerical 
analysis, approximation theory, etc. are presented, e.g., in the monographs [l-3,5,7,9- 
11,14,16,17,22] among the vast bibliography on the subject. In the recent time, convex 
analysis has become more and more important for applications to many new fields such as 
computational statistics, machine learning, and sparse optimization. Having this in mind, 
our major goal here is to revisit the convex subdifferential and provide an easy way to excess 
basic subdifferential calculus rules in finite dimensions. 

In this paper, which can be considered as a supplement to our recent book [14], we develop a 
a geometric approach to convex subdifferential calculus. Our approach relies on the normal 
cone intersection rule based on convex separation and derives from it the major rules of 
subdifferential calculus without any appeal to duality, directional derivatives, and other 
tangentially generated constructions. This approach allows us to give direct and simple 
proofs of known results of convex subdifferential calculus in full generality and also to 
obtain some new results in this direction as those presented in Sections 9, 11, and 12. 
The developed approach is largely induced by the dual-space geometric approach of general 
variational analysis based on the extremal principle for systems of sets, which can be viewed 
as a variational counterpart of convex separation without necessarily imposing convexity; 
see [13] and the references therein. 

Some of the results and proofs presented below have been outlined in the exercises of 
our book [14] while some other results (e.g., those related to subgradients of the optimal 
value function, coderivatives and their applications, etc.) seem to be new in the convex 
settings under consideration. In order to make the paper self-contained for the reader’s 
convenience and also to make this material to be suitable for teaching, we recall here some 
basic definitions and properties of convex sets and functions with illustrative figures and 
examples. Our notation follows [14]. 


2 Basic Properties of Convex Sets 

Here we recall some basic concepts and properties of convex sets. The detailed proofs of 
all the results presented in this and the next section can be found in [14]. Throughout the 
paper, consider the Euclidean space M” of n—tuples of real numbers with the inner product 

n 

{x, y) := ^ XiVi for x = (xi,..., Xn) G M” and y = (yi, ...,yn)& M”. 
i=l 

The Euclidean norm induced by this inner product is defined as usual by 


||x 

We often identify each element x = (xi,..., x„) G M”' with the column x = [xi,..., Xn]^ ■ 
Given two points a,b (z M"", the line segment/interval connecting a and b is 

[a, b] := {Ao + (1 — A)6 | A G [0,1]}. 
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A subset Q of M” is called convex if Ax + (1 — \)y G H for all x,y € Q and A G (0,1). A 
mapping B: M” —>■ MP is affine if there exist apxn matrix A and a vector 6 G such that 

B{x) = Ax + b for all x G M"". 

It is easy to check that the convexity of sets is preserved under images of affine mappings. 


Proposition 2.1 Let B : R"" ^ R^ be an affine mapping. The following properties hold: 

(i) If Ll is a convex subset o/R”, then the direct image B{Ll) is a convex subset o/R^. 

(ii) IfQ is a convex subset o/R^, then the inverse image B ^(0) is a convex subset o/R”. 


For any collection of convex sets their intersection Plie/is also convex. This 

motivates us to define the convex hull of a given set II C R”' by 


co(0) := Pile* 


C is convex and II C C 


i.e., the convex hull of a set H is the smallest convex set containing fl. The following useful 
observation is a direct consequence of the definition. 


Proposition 2.2 For any subset H o/R”, its convex hull admits the representation 

m m 

co(n) = I ^ \iWi I ^ Aj = 1, Aj > 0, rcj G n, m G n|, 

i=l i=l 

where the symbol N stands for the set of all positive integers. 


Given two points a,b ^ R”, the line connecting a and b in R”’ is defined by 

L[a,b] := {Aa + (1-A)6| A G R}. 

A subset A of R” is called affine if for any x,y (z A and for any A G R we have 

Ax + (1 - X)y G A, 

which means that A is affine if and only if the line connecting any two points a, 6 G A is a 
subset of A. This shows that the intersection of any collection of affine sets is an affine set 
and thus allows us to define the affine hull of Ll by 

aff(fl) := P I A I A is affine and II C a|. 

Similarly to the case of the convex hull, we have the following representation. 


Proposition 2.3 For any subset Ll o/R”, its affine hull is represented by 

m m 

aff(n) = I AjWj I Aj = 1 Wj G n, m G n|. 

i=l i=l 
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Figure 1: Affine hull. 


Now we present some simple facts about affine sets. 

Proposition 2.4 Let A be an affine subset of ML. The following properties hold: 

(i) If A contains 0, then it is a subspace o/M”. 

(ii) A is a closed, and so the affine hull of an arbitrary set n is always closed. 

Proof, (i) Since A is affine and since 0 € A, for any x € A and A G M we have that 
Ax = Ax + (1 — A)0 G A. It also holds 

X + y = 2(x/2 + y/2) G A 

for any two elements x,y ^ A, and thus A is a subspace of M”. 

(ii) The conclusion is obvious if A = 0. Suppose that A 7 ^ 0, choose xq G A, and consider 
the set L := A — xq. Then L is affine with 0 G L, and so it is a subspace of M”. Since 
A = xo + T and any subspace of M” is known to be closed, the set A is closed as well. □ 

We are now ready to formulate the notion of the relative interior ri(n) of a convex set 
n C M”, which plays a central role in developing convex subdifferential calculus. 

Definition 2.5 We say that x G ri(n) if there exists 7 > 0 such that 

]B(x; 7 ) n aff(fl) C Q, 

where ]B(x; 7 ) denotes the closed ball centered at x with radius 7 . 

The following simple proposition is useful in what follows and serves as an example for 
better understanding of the relative interior. 
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Figure 2: Relative interior. 


Proposition 2.6 Let LI be a nonempty convex set. Suppose that x € ri(n) and y € LI. 
Then there exists t > 0 such that 


X + t{x — y) £ LI. 

Proof. Choose a number 7 > 0 such that 

B(x; 7 ) n aff(n) C Lt 

and note that x + t{x — y) = (1 + t)x + {—t)y G aff(fl) for all t G M as it is an affine 
combination of x and y. Select t > 0 so small that x + t{x — y) £ ]B(x; 7 ). Then we have 
X + t{x — y) £ B(3:; 7 ) D aff(n) dLl. □ 

Given two elements a,b £ M”, define the half-open interval 

[o, b) := {ta -|- (1 — I 0 < t < 1 }. 

The following theorem is taken from [20, Theorems 6.1 and 6.2]; see also [14, Theorem 1.72] 
for a direct and detailed proof. 

Theorem 2.7 Let LI be a nonempty convex subset ofW^. Then: 

(i) We have ri(n) / 0 and 

(ii) [ 0 , 6 ) C ri(n) for any a £ ri(n) and b £ LI. 

The next theorem taken from [20, Theorem 6.7] gives us a convenient way to represent 
the relative interior of the direct image of a convex set under an affine mapping via of the 
relative interior of this set; see also [14, Excercise 1.27] and its solution for a simple proof. 
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Theorem 2.8 Let B : M”' —>■ be affine, and let C M"' be convex. Then we have 

B(ri(0)) =Ti{B{Q)). 

A useful consequence of this resnlt is the following property concerning the difference of 
two subsets Ai,A 2 C M” defined by 

Ai — A 2 := {oi — 02 I oi € and 02 G ^ 21 - 

Corollary 2.9 Let rii and ^2 ba convex subsets o/M”. Then 

ri(ni - 172 ) = ri(lli) - n{Ll 2 )- 

Proof. Consider the linear mapping B: M” x M"' ^ M” given by B{x, y) := x — y and form 
the Cartesian product Ll := Lli x Ll 2 - Then we have B{Ll) = fli — ^ 2 , which yields 

ri(ffi — ^ 2 ) = A{B{Ll)) = B{m{Ll)) = i?(ri(ni x ^ 2 )) = -B(ri(ffi) x ri(n 2 )) = i'i(fli) — ri(n 2 ) 

by using the simple fact that ri(ffi x ^ 2 ) = ri(fli) x ri(fl 2 ). □ 

Given now a set Ll C M"", the distance function associated with Ll is defined on M"" by 

d{x; LI) := inf{||x — a;|| | w G 0} 

and the Euclidean projection of x onto LI is 

7r(x; n) := {a; G I ||a; — a;|| = d(x; ff)}. 

It is well known (see, e.g., [14, Corollary 1.76]) that 7r(x;ll) is a singleton whenever the set 
n is nonempty, closed, and convex in M”. 

The next proposition plays a crucial role in proving major results on convex separation. 

Proposition 2.10 Let LI be a nonempty closed convex subset o/K"' with x ^ LI. Then we 
have Co = 7r{x; Lt) if and only if Co £ LI and 

{x — Co, to — Co) <0 for all to £ Lt. (2-1) 

Proof. Let ns first show that (2.1) holds for Co := tt{x] 11). Fixing any t £ (0,1) and a; G 11, 
we get la; + (1 — t)Co £ H, which implies by the projection definition that 

||x — Co\\^ < \\x — [too + (1 — l)7;]|p. 

This tells us by the construction of the Enclidean norm that 

||x — a)||^ < Ijx — [a; + l(a; — d;)||^ = jjx — Co\\^ — 2t{x — Co,uj — Co) + t^\\L 0 — d;||^ 

and yields therefore the ineqnality 

2{x — Co ,00 — Co) < l||a; — a)|p. 

Letting there 1 —)• O'*' justifies property (2.1). 

Conversely, suppose that (2.1) is satisfied for a) G H and get for any to £ LI that 

||x —a;|| = jjx — a; + a; — a;|| =||x —a;|| + 2(x — a;,a; — a;) + jja; — a;|| >||x —a;|| . 
Thus we have jjx — a;|| < ||a: — a;|| for all a; G H, which verifies Co = 7r(x; H). □ 
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3 Basic Properties of Convex Functions 

In this section we deal with extended-real-valued functions /: M"’ —>■ (—00,00] = MU {00} 
and use the following arithmetic conventions on (—00,00]: 

a + 00 = 00 + a = 00 for all a G M, 
a • 00 = 00 • a = 00 for all a > 0 , 

00 + 00 = 00, 00 - 00 = 00, 0 -oo = oo -0 = 0 . 

The domain and epigraph of /: R"' —>■ (—00,00] are dehned, respectively, by 

dom(/) := {x G R” I f{x) < 00}, epi(/) := {(x,a) G R”+^ | x G R”, a > /(x)}. 

Recall that a function /: R” —>■ (—00, 00] is convex on R” if 

/(Ax + (1 — A)y) < A/(x) + (1 — A)/(y) for all x,yGR"' and Ag( 0 , 1 ). 

It is not hard to check that / is convex on R"' if and only if its epigraph is a convex subset 
of Furthermore, the domain of a convex function is a convex set. 

The class of convex functions is favorable for optimization theory and applications. The 
next proposition reveals a characteristic feature of convex functions from the viewpoint of 
minimization. Recall that / has a local minimum at x G dom(/) if there is 7 > 0 such that 

f{x) > /(x) for all X G ]B(x;7). 

If this inequality holds for all x G R"", we say that / has an absolute/global minimum at x. 

Proposition 3.1 Let f : R” —>■ (—00, 00] be a convex function. Then f has a local minimum 
at X if and only if f has an absolute minimum at x. 

Proof. We only need to show that any local minimizer of / provides a global minimum 
to this function. Suppose that x is such a local minimizer, fix any x G R”, and denote 
Xk ■= (1 — k~^)x + k~^x for all A: G N. Then x^ —>■ x as A: —>■ 00. Taking 7 > 0 from the 
dehnition of x gives us that x^ G B(x;7) when k is sufficiently large. Hence 

fix) < /(xfc) < (1 - k~/fix) + k~^f{x), 

which readily implies that /(x) < /(x) whenever x G R”'. □ 

Next we present the basic dehnition of the subdijferential as the collection of subgradients 
for a convex function at a given point. 

Definition 3.2 A vector v G R"' is a subgradient of a convex function f: R” ^ (—00,00] 
at X € dom(/) if it satisfies the inequality 

fix) > fix) + (x, X — x) for all x G R"". 

The collection of subgradients is called the subdifferential of f at x and is denoted by dfix). 
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y = f(x) 




/ y = f(x) + <v,x-)c» 





X 


Figure 3: Subgradient. 


Directly from the definition we have the subdifferential Fermat rule: 

f has an absolute minimum at x € dom(/) if and only if 0 G df{x). (3-1) 

Recall that /: M”' —>■ (— 00 , 00 ] is (Frechet) differentiable at x G dom(/) if there is a vector 
V G MF for which we have 

/(x) - f{x) - {v,x- x) ^ ^ 
x^x ||x — x|| 

In this case the vector v is unique, is known as the gradient of / at x, and is denoted by 
V/(x). The next proposition shows that the subdifferential of a convex function at a given 
point reduces to its gradient at this point when the function is differentiable. 

Proposition 3.3 Let f: M"" —>■ (— 00 , 00 ] is convex and differentiable at x ^ dom(/). Then 
(V/(x), X — x) </(x) —/(x) for all x G M” with 9/(x) = {V/(x)}. (3.2) 

Proof. Since / is differentiable at x, for any e > 0 there exists 7 > 0 such that 

—e||x — x|| < /(x) — /(x) — (V/(x), X — x) < e||x — x]| whenever ||x — x|| < 7 . 

Define further the function 

V’(x) := /(x) — /(x) — (V/(x), X — x) + e||x — x|| 

for which '0(x) > V’(^) = 0 whenever x G B(x; 7 ). It follows from the convexity of if that 
fix) > fj{x) when x G M”, and thus 


(V/(x),x — x) </(x) —/(x) + e||x — x|| for all x G M”. 






Letting now e —>■ 0+ gives us the inequality in (3.2) and shows that V/(x) G df{x). 

To verify the remaining part of (3.2), pick any v G df{x) and observe that 

{v, X — x) < f{x) — f{x) for all x G M”. 

From the differentiability of / at x we have 

{v — V/(x), X — x) < e||x — x|| whenever ||x — x|| < 7 , 

and so ||u — V/(x)|| < e. This yields v = V/(x) since e > 0 is arbitrary and thus justifies 
the claimed relationship df{x) = {V/(x)}. □ 

The following simple example calculates the subdifferential of the Euclidean norm function 
directly from the subdifferential definition. 


Example 3.4 For the Euclidean norm function p{x) := ||x|| we have 

f B if X = 0 , 

otherwise, 

X 

where B stands for the closed unit ball of M"'. To verify this, we observe Vp(x) = -r—^ for 

||x|| 

X 7 ^ 0 due to the differentiability of p{x) at nonzero points. Consider the case where x = 0 
and use the definition describe v G dp{0) as 


{v, x) = (u, X — 0) < p{x) — p(0) = ||x|| for all x G M"". 

Eor X = V therein we get {v,v) < ||u||, which shows that ||u|| < 1, i.e., u G B. Conversely, 
picking u G B and employing the Cauchy-Schwarz inequality tell us that 


{v, X — 0) = {v, x) < ||u|| • ||x|| < ||x|| = p{x) — p{0) for all x G M"", 
i.e., V G dp{0). Thus we arrive at the equality dp{0) = B. 


We conclude this section by the useful description of the relative interior of the graph of a 
convex function via the relative interior of its domain; see, e.g., [ 8 , Proposition 1.1.9]. 

Proposition 3.5 For a convex function f: M"' ^ (— 00 , 00 ] we have the representation 
ri(epi(/)) = {( 3 :,A) | x G ri(dom(/)), A > /(x)}. 


4 Convex Separation 

Separation theorems for convex sets, which go back to Minkowski [12], have been well 
recognized among the most fundamental geometric tools of convex analysis. In this section 
we formulate and give simple proofs of several separation results needed in what follows 
under the weakest assumptions in finite dimensions. Let us begin with strict separation of 
a closed convex set and a point outside the set. 
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Proposition 4.1 Let LI be a nonempty closed convex set, and let x ^ LI. Then there exists 
a nonzero vector v G M"’ such that 

sup{(?;,x) I X G il} < {v,x). 


Proof. Denote u := 7r(x; Ll), v := x — u, and fix x G D. Proposition 2.10 gives us 

{v,x — Co) = {x — u),x — uj) < 0, 


which shows that 


{v,x — Co) = {v,x — X + X — Co) = {v,x — X + v) < 0. 
The last inequality therein yields 

{v, x) < {v, x) — ||u||^, 


which implies in turn that 

supKxjx) I X G D} < {v,x) 
and thus completes the proof of the proposition. 


□ 


Remark 4.2 It is easy to see that the closure D of a convex set D is convex. If D C M” is 
a nonempty convex set with x ^ Ll, then applying Proposition 4.1 to the convex set Ll gives 
us a nonzero vector v G M” such that 

sup{(x,x) I X G D} < sup{(x,x) I X G D} < {v,x). 



Figure 4: Separation in a subspace. 


The next proposition justifies a strict separation property in a subspace of 

Proposition 4.3 Let L be a subspace o/M”, and let Ll G L be a nonempty convex set with 
X G T and x ^ Ll. Then there exists v £ L, v ^ 0, such that 

sup{(x,x) I X G D} < {v,x). 


10 







Proof. Employing Remark 4.2 gives us a vector w G ffi” such that 

sup{(rc,x) I X G 0 } < {w,x). 

It is well known that can be represented as the direct sum M"’ = L © L-*-, where 

L"*" := {u € R"' I {u,x) = 0 for all x G L}. 

Thus w = u + V with u G L-^ and v G L. This yields (tt, x) = 0 for any x G 0, C L and 

{v, x) = (u, x) + (u, x) = {u + V, x) = {w, x) < supKtn, x) | x G 12} 

< {w, x) = {u + V, x) = {u, x) + {v, x) = (u, x), 

which shows that sup{(u,x) | x G 0} < (u,x) with u 7 ^ 0. □ 



Figure 5: Illustration of the proof of Lemma 4.4. 


Lemma 4.4 Let Ll C R” be a nonempty convex set, and let 0 G n\ri(II). Then aff(n) is a 
subspace ofET, and there is a sequence {xk} C aff(r2) with Xk and Xfc —>• 0 as k ^ 00 . 

Proof. Since ri(n) 7 ^ 0 by Theorem 2.7(i) and 0 G H \ ri(II), we find xq G ri(n) and 
conclude that —txo ^ for all t > 0. Indeed, suppose by contradiction that —txo G fl for 
some t > 0 and then deduce from Theorem 2.7(ii) that 

0 = e ri(Q), 

which contradicts 0 ^ ri(n). Letting now Xk '■= implies that Xk ^ Ll for every k and 
Xfc —)• 0 as A: —>■ 00 . Furthermore, we have 

0 G n C aff(r2) = aff(r2) 
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by the closedness of aff(n) due to Proposition 2.4(ii). This shows that aff(n) is a subspace 
and that G aff(n) for all A; G N. □ 

We continue with another important separation property known as proper separation. 

Definition 4.5 It is said that two nonempty convex sets fii and 11,2 properly separated 
if there exists a nonzero vector v G M” such that 

sup{(n,a:) | x G Sdi} < inf{(n,y) | y G 02 }, inf{(n,a;) | x G fii} < sup{{v,y) \ y G 122 }- 

Lemma 4.6 Let LI be a nonempty convex set in M”'. Then 0 ^ ri(n) if and only if the sets 

and { 0 } are properly separated, i.e., there is v £ M”', n 7 ^ 0 , such that 

sup{(n,x) I x G 0} < 0, inf{(n, 3 :) | x G 0} < 0. 

Proof. We split the proof into the following two cases. 

Case 1: 0 ^ LI. It follows from Remark 4.2 with x = 0 that there exists n 7 ^ 0 such that 

supK^jx) I a; G 0} < (n, x) = 0, 
and thus the sets 12 and { 0 } are properly separated. 

Case 2: 0 G 12 \ ri(I2). Letting L := aff(f2) and employing Lemma 4.4 tell us that L is a 
subspace of and there is a sequence {xk} C L with Xk ^ Ll and Xfc —>■ 0 as A; —>■ 00 . By 
Proposition 4.3 there is a sequence {vk} C L with Ufc 7 ^ 0 and 

sup{(nfc,x) I X G f2} < {vk,Xk), A: G N. 

Denoting '■= ||^ shows that ||u;fc|| = 1 for all A: G N and 

{wk,x) < {wk,Xk) for all x G 12. (4.1) 

Letting A: ^ 00 in (4.1) and supposing without loss of generality that Wk ^ v £ L with 
some ||n|| = 1 along the whole sequence of {A:}, we arrive at 

sup{(n,x) I X G 12} < 0 

by taking into account that \{wk,Xk)\ < = ||xfc|| —)■ 0. To verify further 

infKujx) I X G 12} < 0, 

it suffices to show that there is x G 12 with {v, x) < 0. Suppose by contradiction that 
(n,x) > 0 for all X G 12 and deduce from sup{(u,x) | x G 12} < 0 that (u,x) = 0 for all 
X G 12. Since v £ L = aff(12), we get the representation 

m m 

V = AjWj with Aj = 1 and uji £ LI for i = 1,.. . ,m, 
i=l i=l 
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which readily implies the equalities 

m 

||u|p = {v,v) = '^Xi{v,uJi) = 0 
i=l 

and so contradicts the condition ||u|| = 1. This justifies the proper separation of n and {0}. 

To verify the reverse statement of the lemma, assume that n and {0} are properly separated 
and thus find 0 7 ^ u € M” such that 

supKujx) I a; G 11 } < 0 while {v,x) < 0 for some x €Cl. 

Suppose by contradiction that 0 G ri(H) and deduce from Proposition 2.6 that 

0 +1{0 — x) = —tx G n for some t > 0. 

This immediately implies the inequalities 

{v,—tx) < supKujx) I X G H} < 0 

showing that {v,x) > 0. It is a contradiction, which verifies 0 ^ ri(fl). □ 

Now we are ready to prove the main separation theorem in convex analysis. 

Theorem 4.7 Let Hi and H 2 be two nonempty convex subsets of ML. Then Hi and H 2 are 
properly separated if and only f/ri(Hi) nri(H 2 ) = 0. 

Proof. Define H := Hi — H 2 and verify that ri(Hi) n ri(H 2 ) = 0 if and only if 

0 ri(Hi — H 2 ) = ri(Hi) — ri(H 2 ). 

To proceed, suppose first that ri(Hi) n ri(H 2 ) = 0 and so get by Corollary 2.9 that 0 0 
ri(Hi — H 2 ) = ri(H). Then Lemma 4.6 tells us that the sets H and {0} are properly 
separated. Thus there exist v G M” with (u, x) < 0 for all x G H and also y ^ LI such that 
(u, y) < 0. For any wi G Hi and 0^2 G H 2 we have x := wi — a ;2 G H, and hence 

{v,uji - UJ 2 ) = (u,x) < 0 , 

which yields {v,uji) < (u, W 2 ). Choose a)i G Hi and W 2 G H 2 such that y = uii — ui 2 . Then 

(ujWi - (L 2 ) = {v,y) < 0 

telling us that {v,u)i) < {v,u) 2 )- Hence Hi and H 2 are properly separated. 

To prove next the converse implication, suppose that Hi and H 2 are properly separated, 
which implies that the sets H = Hi — H 2 and {0} are properly separated as well. Employing 
Lemma 4.6 again provides the relationships 

0 ^ ri(H) = ri(Hi — H 2 ) = ri(Hi) — ri(H 2 ) and so ri(Hi) n ri(H 2 ) = 0, 

which thus complete the proof of the theorem. □ 

Among various consequences of Theorem 4.7, including those presented below, note the 
following relationships between the closure and relative interior operations on convex sets 
that seemingly have nothing to do with separation. 


13 


Corollary 4.8 (i) // O C M” is convex, then ri(n) = ri(n) and ri(0) = 0. 

(ii) If both sets ^1,02 C M”' are convex, then we have ri(ili) = ri(02) provided that fli = 02- 

Proof, (i) Both equalities are trivial if = 0. To verify the first equality in (i) when 
n 7 ^ 0 , observe that for any x G we have the equivalence 

{x} and n are properly separated {x} and Q are properly separated. 

Indeed, the implication “^=>” is obvious because for any v G M” 

sup{(u, re) I re G n} < {v, x)], 

which can be proved by a limiting argument. For the converse, suppose that {x} and H are 
properly separated. Then there exists v G M”, v ^ 0, such that 

sup{(x,u;) I u; G 0} < (x,x), inf{(x,tc) | tc G < {v,x). 

It follows that 

supKx,?!;) I re G II} < sup{(x,t(;) | tc G H} < {v,x). 

It remains to show that there exists w € Q such that {v,w) < {v,x). If this is not the case, 
then {v, w) > {v, x) for all w € H, which implies {v, w) = {v, x) for all w £ H. A simple 
limiting argument yields {v,w) = {v,x) for all w £ H. This is a contradiction because 
inf{(x,t(;) I tc G n} < (x,x). 

Defining further 0 := {x}, we get ri(0) = {x} and deduce from Theorem 4.7 that 

X ^ ri(II) ri(0) D ri(II) = 0 

{x} and D are properly separated 
{x} and D are properly separated 
ri( 0 ) n ri(II) = 0 x ^ ri(D). 

The second equality in (i) is a direct consequence of Theorem 2.7(ii). 

(ii) If Di = D 2 , then ri(IIi) = ri(IIi) and hence ri(Di) = ri(Il 2 ) by (i). □ 

5 Normal Cone Intersection Rule 

In this section we derive the central result of the geometric approach to convex subdifferen¬ 
tial calculus, which provides a general intersection rule for the normal cone to convex sets. 
All the subsequent subdifferential results are consequences of this intersection rule. 

Recall first the definition of the normal cone to a convex set. 

Definition 5.1 Let LI be a nonempty eonvex subset of ML. Then the normal cone to the 
set LI at x £ LI is defined by 

N{x; Ll) := {x G M” | (x, x — x) < 0 for all x £ D}. 

In the case where x ^ LI we define N{x; D) := 0. 


[sup{(x,r(;) I rc G D} < (x,x)] 
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Figure 6 : Normal cone. 


It immediately follows from Definition 5.3 that N{x; D) is a closed and convex cone, which 
reduces to {0} if x € int(D). A remarkable property of the normal cone to a convex set 
D in finite dimensions is that N(x-,Q) ^ {0} if and only if x is boundary point of D; see, 
e.g., [14, Corollary 2.14]. This is the classical supporting hyperplane theorem, which can be 
easily derived by the limiting procedure from Theorem 4.7. 

Before deriving our major intersection result on the representation of the normal cone to 
finitely many convex sets, let us present a useful lemma on the relative interior of set 
intersections, which is also based on convex separation. 

Lemma 5.2 Let Dj C M” for i = 1,... ,m with m >2 be convex subsets of M"" such that 

m 

P|ri(Di)/0. (5.1) 

i=l 

Then we have the representation 

m m 

ri(^ Pi = Pi ri(Di). (5.2) 

2=1 2=1 

Proof. We first verify this result for m = 2. Pick x G ri(Di) D ri(D 2 ) and find 7 > 0 with 
B(x; 7) n aff(Di) C and B(x; 7) D aff(D2) C D 2 , 
which implies therefore that 

B(x; 7) n [aff(Di) n aff(D 2 )] C n D 2 . 
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It is easy to see that aff(IIi n 02 ) C aff(fli) D aff(r 22 ), and hence 

]B(x; 7 ) n aff(rii n 112 ) c Hi n H 2 . 

Thus we get x G ri(Hi n H 2 ), which justifies that ri(Hi) n ri(H 2 ) C ri(Hi n H 2 ). 

To verify the opposite inclusion in (5.2) for m = 2, observe that 

Hi n H 2 = Hi n H 2 (5'3) 

for any convex sets Hi, H 2 with ri(Hi) n ri(H 2 ) 7 ^ 0. Indeed, pick x G Hi n H 2 , x G ri(Hi) n 
ri(H 2 ) and observe that Xk ■= k~^x->r (1 — k~^)x —>■ x as /c —)• 00 . Then Theorem 2.7(ii) tells 
us that Xfc G Hi n H 2 for large A: G N and hence x G Hi n H 2 , which justifies the inclusion 
“D” in (5.3). The inclusion “c” therein obviously holds even for nonconvex sets. Now using 
(5.3) and the second equality in Corollary 4.8(i) gives us 

ri(Hi) n ri(H 2 ) = ri(Hi) n ri(H 2 ) = Hi n H 2 = Hi n H 2 . 

Then we have the equality 

ri(ri(Hi) n ri(H 2 )) = ri(Hi D H 2 ), 
and thus conclude by Corollary 4.8(ii) that 

ri(ri(Hi) n ri(H 2 )) = ri(Hi n H 2 ) C ri(Hi) D ri(H 2 ), 
which justify representation (5.2) for m = 2. 

To verify (5.2) under the validity of (5.1) in the general case of m > 2, we proceed by 
induction with taking into account that the result has been established for two sets and 
assuming that it holds for m — 1 sets. Considering m sets Hj, represent their intersection as 

m m—1 

Hj = H n Hm with H := Hj. (5-4) 

i=l i=l 

Then we have ri(H) n ri(Hm) = n^]^ri(Hj) 7 ^ 0 by the imposed condition in (5.1) and the 
induction assumption on the validity of (5.2) for m — 1 sets. This allows us to employ the 
obtained result for the two sets H and H^ and thus arrive at the desired conclusion (5.2) 
for the m sets Hi,..., H^ under consideration. □ 

Now we are ready to derive the underlying formula for the representation of the normal cone 
to intersections of finitely many convex sets. Note that the proof of this result and of the 
subsequent calculus rules for functions and set-valued mappings mainly follow the geometric 
pattern of variational analysis as in [13]. The specific features of convexity and the usage of 
convex separation instead of the extremal principle allow us to essentially simplify the proof 
and to avoid the closedness requirement on sets and the corresponding lower semicontinuity 
assumptions on functions in subdifferential calculus rules. Furthermore, we show below that 
the developed geometric approach works in the convex setting under the relative interior 
qualification conditions, which are well-recognized in finite-dimensional convex analysis and 
occur to be weaker than the basic/normal qualifications employed in [13,14]; see. e.g.. 
Corollary 5.5 and Example 5.6 below. 
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N{x;Q2) 


N{x; fli) 



Figure 7: Intersection rule. 


Theorem 5.3 Let fli, ..., C M"" be convex sets satisfying the relative interior condition 

m 

Pi ri(ni) / 0, (5.5) 

i=\ 

where m>2. Then we have the intersection rule 

mm m 

n(^x-, Pi N{x; Lli) for all x G P| fij. (5.6) 

i=l i=l i=l 

Proof. Proceeding by induction, let us first prove the statement of the theorem for the 

case of m = 2. Since the inclusion “D” in (5.6) trivially holds even without imposing 

(5.5), the real task is to verify the opposite inclusion therein. Fixing x G fli D and 
X G A^(x; rii n LI 2 ), we get by the normal cone definition that 

(x, X — x) < 0 for all x G n 1 ^ 2 . 

Denote 0i := Di x [0,oo) and ©2 := {(a;,A) | x G LI 2 , A < (x,x — x)}. It follows from 
Proposition 3.5 that ri(0i) = ri(Di) x (0,oo) and 

ri( 02 ) = {(x. A) I X G ri(D 2 ), A < (x, x — x)}. 

Arguing by contradiction, it is easy to check that ri(0i) n ri( 02 ) = 0. Then applying 
Theorem 4.7 to these convex sets in gives us 0 7 ^ {'w,'y) G M"" x M such that 

{w, x) + Ai 7 < (rx, y) + A 27 for all (x, Ai) G ©i, (y, A 2 ) G © 2 . (5.7) 

Moreover, there are (x, Ai) G ©1 and (y, A 2 ) G ©2 satisfying 

{w, x) + Ai 7 < (tx, y) + A 27 . 
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Observe that 7 < 0 since otherwise we can get a contradiction by employing (5.7) with 
{x,k) G ©1 as /c > 0 and (x,0) G 02 - Let us now show by using (5.5) that 7 < 0. Again 
arguing by contradiction, suppose that 7 = 0 and then get 

{w,x) < {w,y) for all x G hli, y G ^2 and {w,x) < {w,y) with x G hli, y G f^ 2 - 

This means the proper separation of the sets hli and Q 2 , which tells us by Theorem 4.7 that 
ri(r2i) n ri(i 42 ) = 0- The obtained contradiction verifies the claim of 7 < 0 . 

To proceed further, denote y, := —7 > 0 and deduce from (5.7), by taking into account that 
(x,0) G ©1 when x G hli and and that (x,0) G © 2 , the inequality 

{w,x) < {w,x) for all x G hli. 

This yields w G N{x]Qi) and hence — G A^(x;ni). Moreover, we get from (5.7), due to 
(x, 0) G ©1 and (y, a) G ©2 for all y & ^2 with a = {v,y — x), that 

(rc, x) < (re, y) + 7 (u, y — x) whenever y G ^ 2 - 

Dividing both sides therein by 7 , we arrive at the relationship 

^— + u, y — x^ < 0 for all y G D 2 , 

w w _ 

and thus-|-u =-huG N{x] D 2 ). This gives us 

7 fJ- 

w 

V £ -h N{x-, D2) C N{x-, Di) + N{x-, D2) 

completing therefore the proof of (5.6) in the case of m = 2. 

Considering now the case of intersections for any finite number of sets, suppose by induction 
that the intersection rule (5.6) holds under (5.5) for m — 1 sets and verify that it continues 
to hold for the intersection of m > 2 sets Represent the latter intersection as 

D n Dm with D := we get from the imposed relative interior condition (5.5) and 

Lemma 5.2 that 

m 

ri(D) n ri(Dm) = Q ri(Dj) ^ 0 . 

i=l 

Applying the intersection rule (5.6) to the two sets DnDm and then employing the induction 
assumption for m — 1 sets give us the equalities 

m m 

n(^x-, Pi Di) = iV(x; D n Dm) = iV(x; D) + N{x; Dm) = N{x; D*), 

i=l i=l 

which thus justify (5.6) for m sets and thus completes the proof of the theorem. □ 

It is not difficult to observe the relative interior assumption (5.5) is essential for the validity 
of the intersection rule (5.6) as illustrated by the following example. 
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Example 5.4 Define the two convex sets on the plane by 

Di := {(x, A) G I A > x^} and D 2 := {(x, A) G | A < —x^}. 

Then for x = (0,0) G Di n D 2 we have 

N{x', Di) = {0} X (— 00 ,0], N{x] D 2 ) = {0} X [0, 00 ), and N{x', n D 2 ) = 

Thus N{x\ Di) + N{x] D 2 ) = {0} x M 7 ^ A^(x; Di n D 2 ), i-e., the intersection rule (5.6) fails. 
It does not contradict Theorem 5.3, since ri(Di) n ri(D 2 ) = 0 and so the relative interior 
qualification condition (5.5) does not hold in this case. 



Figure 8 : Illustration of the relative interior condition. 


Finally, we compare the intersection rule of Theorem 5.3 derived under the relative interior 
qualification condition (5.5) with the corresponding result obtained in [14, Corollary 2.16] 
for m = 2 under the so-called basic/normal qualification condition 

iV(x;Di)n [-iV(x;D 2 )] = {0} (5.8) 

introduced and applied earlier for deriving the intersection rule and related calculus results 
in nonconvex variational analysis; see, e.g., [13,21] and the references therein. Let us first 
show that (5.8) yields (5.5) in the general convex setting. 

Corollary 5.5 Let Di,D 2 C M” be convex sets satisfying the basic qualification condition 
(5.8) at some x G n D 2 . Then we have 

ri(Di) n ri(D 2 ) 0, (5.9) 

and so the intersection rule (5.6) holds for these sets at any x G Di n D 2 - 
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Proof. Arguing by contradiction, suppose that ri(r2i) n ri(r22) = 0- Then the sets r2i,r22 
are properly separated by Theorem 4.7, and so there is u 7 ^ 0 such that 

{v,x) < {v,y) for all x G Cli, y £ fl 2 - 

Since x € ^2, we have (u,a; — x) < 0 for all x € fli. Hence v G N{x]^li) and similarly 
—V G N{x]^}2)- Thus 0 7^ u G N{x]Qi) fl [— A^(x; 02)], which contradicts (5.8). □ 

The next example demonstrates that (5.9) may be strictly weaker then (5.8). 

Example 5.6 Consider the two convex sets on the plane defined by fli := M x {0} and 
kl 2 := (— 00 , 0 ] X {0}. We obviously get that condition (5.9) is satisfied ensuring thus the 
validity of the intersection rule by Theorem 5.3. On the other hand, it follows for that 

A'(a;; rii) = {0} X M and A^(x; ^ 2 ) = [0, 00 ) x M with x = (0,0), 

i.e., the other qualification condition (5.8) fails, which shows that the result of [14, Corol¬ 
lary 2.16] is not applicable in this case. 


6 Subdifferential Sum Rule and Existence of Subgradients 

The main goal of this section is to derive from the geometric intersection rule of Theorem 5.3 
the subdifferential sum rule for convex extended-real-valued functions under the least re¬ 
strictive relative interior qualihcation condition. Then we deduce from it a mild condition 
ensuring the existence of subgradients for general convex functions. 

Prior to this, let us recall well-known relationships between normals to convex sets and 
subgradients of convex functions used in what follows. 

Proposition 6.1 (i) Let Ll C ML be a nonempty convex set, and let 5{x-,Ll) = 6 q{x) be its 
indicator function equal to 0 when x G LI and to 00 otherwise. Then we have 

d5{x; n) = N{x; Ll) for any x gLI. 

(ii) Let f : M” —>■ (—00,00] be a convex function, and let x G dom(f). Then we have 
9 fix) = {u G M*" I (v, -1) G Ni{x, /(x)); epi(/))}. 

Proof, (i) It follows directly from the definitions of the subdifferential, normal cone, and 
the set indicator function. 

(ii) Fix any subgradient v G df{x) and then get from Definition 3.2 that 

{v, X — x) < fix) — fix) for all x G M”. (6.1) 
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To show that {v, —1) € N{{x, /(x)); epi(/)), fix any (x, A) G epi(/) and observe that due to 
A > /(x) we have the relationships 

{{v, -1), {x, A) - {x, f{x))) = {v,x-x) + (-1)(A - f{x)) 

= {v,x - x) - {X- f{x)) < {v,x-x) - (fix) - f{x)) < 0, 

where the the last inequality holds by (6.1). To verify the opposite inclusion in (ii), take 
(u,—1) G N{{x, f (x))] epi{f)) and fix any x G dom(/). Then (x,/(x)) G epi(/) and hence 

{{v, -1), {x, f{x)) - {x, f{x))) < 0, 

which in turn implies the inequality 

{v,x-x) - (fix)- f{x)) < 0. 

Thus V G df{x), which completes the proof of the proposition. □ 

Now we are ready to deduce the following subdifferential sum rule for function from the 
intersection rule of Theorem 5.3 for normals to sets. 

Theorem 6.2 Let fi \ M" —>■ (— 00 , 00 ], f be extended-real-valued convex func¬ 

tions satisfying the relative interior qualification condition 

m 

Pi ri(^dom(/i)^ / 0 , ( 6 . 2 ) 

i=l 

where m>2. Then for all x G dom(/j) we have the sum rule 

m m 

2=1 2 = 1 

Proof. Observing that the inclusion “d” in (6.3) directly follows from the subdifferential 
definition, we proceed with the proof of the opposite inclusion. Consider first the case of 
m = 2 and pick any v G d{fi + f 2 ){x)- Then we have 

{v, X -x) <{fi + f 2 ){x) - (/i + f 2 ){x) for all x G M”. (6.4) 

Define the following convex subsets of by 

Di := {(x, Ai, A 2 ) G M” X M X M I Ai > /i(x)} = epi(/i) x M, 

D 2 := {{x, Ai, A 2 ) G M” X M X M I A 2 > f 2 {x)}- 

We can easily verify by (6.4) and the normal cone definition that 

{v, -1, -1) G N{{x, /i(x), / 2 (x)); n ^ 2 )- 

To apply Theorem 5.3 to these sets, let us check that ri(Di) n ri(D 2 ) 7 ^ 0- Indeed, we get 

ri(Di) = {(x, Ai, A 2 ) G M” x M x M | x G ri(dom(/i)), Ai > /i(x)} = ri(epi(/i) x M, 
ri(D 2 ) = {(x,Ai,A 2 ) GM"xMx]R|xG ri(dom(/ 2 )), A 2 > f 2 {x)} 
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by Proposition 3.5. Then choosing z G ri(dom(/i)) D ri(dom(/ 2 )), it is not hard to see that 

+ l,f2iz) + 1) G ri(fii) nri(i72) / 0- 


Applying now Theorem 5.3 to the above set intersection gives us 



It follows from the structures of the sets and ^2 that 

(u,-!,-!) = (?;i,-7i,0) + (u2,0,-72) 

with (ui,- 7 i) G iV((x,/i(x));epi(/i)) and (^ 2 ,- 72 ) G iV((x,/ 2 (x)); epi(/ 2 )). Thus 

V = Vi+V2, 71=72 = 1 , 

and we have by Proposition 6.1(ii) that vi G dfi{x) and V 2 G < 9 / 2 (®)- This ensures the 
inclusion d{fi + / 2 )(®) C dfi{x) + < 9 / 2 (x) and hence verihes (6.3) in the case of m = 2. To 
complete the proof of the theorem in the general case of m > 2 , we proceed by induction 
similarly to the proof of Theorem 5.3 with the usage of Lemma 5.2 to deal with relative 
interiors in the qualification condition ( 6 . 2 ). □ 

The next result is a simple consequence of Theorem 6.2 providing a mild condition for the 
existence of subgradients of an extended-real-valued convex function at a given point. 

Corollary 6.3 Let f: M"' ^ (— 00 , 00 ] be a convex function. Then the validity of the relative 
interiority condition x G ri(dom/) ensures that df{x) 7^ 0 . 

Proof. Define the extended-real-valued function on M” by 


f{x) if X = x, 
00 otherwise 


g{x) := f{x) + 6^s:}{x) 


via the indicator function of the singleton {x}. Then epi( 5 ') = {x} x [/(x),oo) and hence 


N{{x, g{x));epi{g)) = M” x (— 00 ,0]. We obviously get that dg{x) = M” and that d6{x}{x) = 
N{x] {x}) = ML by Proposition 6.1(i). We further have 


ri(dom(/i)) = {x} for h{x) := 5j^}(x) 


and thus ri(dom(/)) nri(dom(/i)) 7 ^ 0. Applying the subdifferential sum rule of Theorem 6.2 
to the above function g{x) at x gives us 


M” = dg{x) = 5/(x) +M^, 


which justifies the claimed assertion on df{x) 7 ^ 0 . 


□ 
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7 Subdifferential Chain Rule 


In this section we employ the intersection rule of Theorem 5.3 to derive a chain rule of the 
subdifferential of a composition of an extended-real-valued function and an affine mapping 
under which we obviously keep convexity. First we present the following useful lemma. 

Lemma 7.1 Let B : R"" —>■ R^ be an affine mapping given by B{x) := Ax + b, where A is 
a p X n matrix and b G R?*. Then for any {x, y) G gph(B) we have 

N[{x, y); gph(S)) = {(u, u) G R”' x R^ | m = —A~^v}. 

Proof. It is clear that gph(i?) is convex and {u,v) G N{{x,y) gph(i?)) if and only if 

{u,x — x) + {v,B{x) — B{x)) < 0 for all x G R”. (7.1) 

It follows directly from the definitions that 

(n, X — x) + {v,B{x) — B{x)) = {u,x — x) + {v, A{x) — A{x)) 

= {u,x — x) + {Al^ V, X — x) = {u + AJ X — x). 

This implies the equivalence of (7.1) to {u + AJv,x — x) < 0 for all x G R”, and so to 
u = —A~^v. □ 

Theorem 7.2 Let /: R^ —>■ (—00,00] be a convex function, and let B : R”' —R^ be as in 
Lemma 7.1 with B{x) G dom(/) for some x G R”. Denote y := B{x) and assume that the 
range of B contains a point o/ri(dom(/)). Then we have the subdifferential chain rule 

d{f oB){x) = A~^[df{y)) = {A'^v \ v G df{y)}. (7.2) 

Proof. Fix V G d{f o B){x) and form the subsets of R"’ x RP x R by 

rii := gph(i?) X R and LI 2 ■= x epi(/). 

Then we clearly get the relationships 

ri(IIi) = Hi = gph(.B) X R, ri(H 2 ) = {(x,y,A) | x G R”, y G ri(dom(/)), A > f{y)}, 

and thus the assumption of the theorem tells us that ri(Hi) n ri(H 2 ) 7^ 0. 

Further, it follows from the definitions of the subdifferential and of the normal cone that 
(x,0, —1) G A^((x, y, ^); Hi n H 2 ), where 2 := /(y). Indeed, for any (x,y, A) G Hi n H 2 we 
have y = B{x) and A > /(y), and so A > f{B{x)). Thus 

{v, X - x) + d{y -y) + (-1)(A - z) < {v,x - x) - [f{B{x)) - f{B{x))] < 0. 

Employing the intersection rule of Theorem 5.3 to the above sets gives us 

(x,0,-l) G IV((x,y, A);Hi) -b iV((x, y, 7); H 2 ), 
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which reads that (f,0, —1) = tc,0) + (0,tc,—1) with (vj—w) G N{{x,y);gph{B)) and 

(w,-l) € N{{y,z);epi{f)). Then we get 

V = w and w G df{y), 

which implies in turn that v G {df{y)) and hence verifies the inclusion “c” in (7.2). The 
opposite inclusion follows directly from the definition of the subdifferential. □ 


8 Subdifferentiation of Maximum Functions 

Our next topic is subdifferentiation of an important class of nonsmooth convex functions 
dehned as the pointwise maximum of convex functions. We calculate the subdifferential of 
such functions by using again the geometric intersection rule of Theorem 5.3. 

Given /*: M"’ —>■ (—oo, oo] for i = 1,... ,m, define the maximum function by 

f{x) := max fi{x), x G M", (8.1) 

and for x G M” consider the active index set 

I{x) := {z G {1,... ,m} I fi{x) = f{x)}. 

Lemma 8.1 (i) Let LI be a convex set in Then int (Ll) = ri(f2) provided that int (Ll) ^ 0. 
Furthermore, N{x]Ll) = {0} if x G int(fl). 

(ii) Let f: M” — (— 00 , 00 ] be a convex function, which is continuous at x € dom(/). Then 
we have x G int(dom(/)) with the implication 

{v,-X) G iV((x,/(x));epi(/)) ^ [A > 0 and u G Xdf{x)]. 

Proof, (i) Suppose that int(f2) 7 ^ 0 and check that aff(fl) = M"'. Indeed, picking x G int(n) 
and fixing x G M”, find t > 0 with tx + {1 — t)x = x + t{x — x) G int(n) C aff(n). It yields 

X = + (1 — t)x) + (1 ~ -)x G aff(n), 

which justifies the claimed statement due to the definition of relative interior. 

To verify the second statement in (i), take v G N{x‘, Ll) with x G int (II) and get 

{v,x — x) <0 for all x € Ll. 

Choosing 5 > 0 such that x + tv € B(x; d) C II for t > 0 sufficiently small, gives us 

{v,X + tv — x) = tllulP < 0, 

which implies u = 0 and thus completes the proof of assertion (i). 

(ii) The continuity of / allows us to find d > 0 such that 

\f{x) — f{x)\ < 1 whenever x G ]B(x; 5). 
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This yields ]B(x;5) C dom(/) and shows therefore that x G int(dom(/)). 

Now suppose that {v,—X) G N{{x, f{x))]epi{f)). Then 

{v,x — x) — X{t — f{x)) < 0 whenever {x,t) G epi(/). (8.2) 

Employing this inequality with x = x and t = f{x) + 1 yields A > 0. 

If A > 0, we readily get (n/A,—1) G A^((x,/(x));epi(/)). It follows from Proposition 6.1 
that v/X € df{x), and hence v G Xdf{x). 

In the case where A = 0, we deduce from (8.2) that v € N{x] (dom(/)) = {0}, and so the 
inclusion v G Xdf{x) is also valid. Note that df{x) 7^ 0 by Corollary 6 . 3 . □ 

Now we are ready to derive the following maximum rule. 

Theorem 8.2 Let fi \ M” —>■ (— 00 , 00 ], i = he convex functions, and let x G 

n™ 1 dom/j be such that each fi is continuous at x. Then we have the maximum rule: 

5(max/i)(x) = CO I |J dfi{x) 

Proof. Let / be the maximum function defined in (8.1) for which we obviously have 

m 

epi(/) = P|epi(/i). 
i=l 

Employing Proposition 3.5 and Lemma 8.1(i) give us the equalities 

ri(epi(/i)) = {(x,A) | x G ri(dom(/i)), A > fi{x)} = {(x,A) | x G int(dom(/i)), A > /i(x)}, 

which imply that (x,/(x) + 1) G ™t(epi(/i)) = Furthermore, since 

fi{x) < f{x) = a for any i ^ I{x), there exists a neighborhood U oi x and 7 > 0 such that 
fi{x) < a whenever (x,a) G t/ x (a —7,0 + 7). It follows that (x,d) G int(epi(/i)), and so 
N{{x,a);epi{fi)) = {(0,0)} for such indices i. Thus Theorem 5.3 tells us that 

m 

N{{x,f{x))-,epi{f))='^N{{x,a)]ep\{fi))= ^ A^((x,/i(x)); epi(/i)). 

i=l iGl(x) 

Picking now v G df{x), we have by Proposition 6.1(ii) that (u,—1) G N((x, f(x)}; epif), 
which allows us to find {vi,—Xi) G N{{x, fi{x))] epifi) for i G I{x) such that 

(^^,-1) = 

i€l{x) 

This yields = 1, A* > 0, u = ^ hdfi{x) by Lemma 8.1(ii). 

Thus V = XiUi, where Ui G dfi{x) and Yliei{x) = ^- This verifies that 

G CO ( [J dfi{x) 

\iei{x) 
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The opposite inclusion in the maximum rule follows from 

dfi{x) C df{x) for all i ^ I{x), 

which in turn follows directly from the definitions. □ 


9 Optimal Value Function and Another Chain Rule 

The main result of this section concerns calculating the subdifferential of extended-real- 
valued convex functions, which play a remarkable role in variational analysis, optimization, 
and their numerous applications and are known under the name of optimal value/marginal 
functions. Functions of this class are generally defined by 

p,{x) :=mi{ip{x,y)\y e F{x)}, (9.1) 

where (p: M” x ^ (~oo, oo] is an extended-real-valued function, and where F: M” 
is a set-valued mapping, i.e., F{x) C for every x G M”. In what follows we select p and F 
in such a way that the resulting function (9.1) is convex and to derive a formula to express 
its subdifferential via the subdifferential of p and an appropriate generalized differentiation 
construction for the set-valued mapping F. The results obtained in the general framework 
of variational analysis [13, 21] advise us that the most suitable construction for F for these 
purposes is the so-called coderivative of F at (x, y) G gph(T) defined via the normal cone 
to the graphical set gph(F) := {{x,y) G M"’ x | y G F{x)} by 

D*F{x,y){v) = {u€R^\{u,-v)eN{{x,y);gph{F))}, u G (9.2) 

It is easy to check that the optimal value function (9.1) is convex provided that p is convex 
and the graph of F is convex as well. An example of such F is given by the affine mapping 
B considered in Lemma 7.1. Note that, as follows directly from Lemma 7.1 and definition 
(9.2), the coderivative of this mapping is calculated by 

D*B{x,y){v) = A^v with y = B{x). (9.3) 

Now we proceed with calculating the subdifferential of (9.1) via that of p and the coderiva¬ 
tive of F. The results of this type are induced by those in variational analysis [13,21], 
where only upper estimate of dp,{x) were obtained. The convexity setting of this paper 
and the developed approach allow us to derive an exact formula (equality) for calculating 
dp{x) under a mild relative interior condition, which is strictly weaker than the normal 
qualification condition from [14, Theorem 2.61]; cf. the discussion at the end of Section 5. 

Theorem 9.1 Let y{-) be the optimal value function (9.1) generated by a convex-graph 
mapping F : RF and a convex function p : R” x R^ —(—00,00]. Suppose that 

p{x) > —00 for all X G R”, fix some x G dom(/i), and consider the solution set 

S{x) := {y G F{x) \ p{x) = p{x,y)]. 
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If S{x) / 0, then for any y G S{x) we have the equality 


dy{x)= [J [u +D*F{x,y){v)] 

{u,v}^dip{x,y) 

provided the validity of the relative interior qualification condition 

ri(dom(<^)) nri(gph(F)) / 0. 


(9.4) 


(9.5) 


Proof. Picking any y € S{x), let us first verify the estimate 

[u +D*F{x,y){v)] C dfi{x). (9.6) 

{u,v}&dip{x,y) 


To proceed, take w from the set on the left-hand side of (9.6) and find {u,v) G dip{x,y) 
with w — u G D*F{x, y){v). It gives us {w — u, —v) G N{{x, y); gph(F)) and thus 

{w — u, X — x) — {v, y — y) <0 for all (x, y) G gph(T), 

which shows that whenever y G F{x) we have 

{w, x - x) < {u,x - x) + {v,y - y) < ip{x, y) - ip{x, y) = <y9(x, y) - y{x). 

This allows us to arrive at the estimate 

{w, X — x) < inf ip{x, y) — fi{x) = y{x) — y{x) 
yeFix) 

justifying the inclusion w G diJ,{x) and hence the claimed one in (9.6). 

It remains to verify the inclusion “c” in (9.4). Take w G dy{x), y G S{x) and get 

{w, x-x) < y,{x) - y,{x) = y{x) - (/?(x, y) < ip{x, y) - ip{x, y) 

whenever y G F(x) and x G M"". This implies in turn that for any (x, y) G M"" x we have 

(w,x- x) + (0,y-y) < (p(x,y) + S((x,y);gph(F)) - [(p(x,y) + d((x,y);gph(F))]. 

Considering further f{x,y) := ip{x,y) + (I((x, y); gph(F)), deduce from the subdifferential 
sum rule of Theorem 6.2 under (9.5) that 

{w, 0) G df{x, y) = dip{x, y) + A^((x, y); gph(F)). 

This shows that (u), 0) = (ui, ui)+(u 2 , U 2 ) with {ui,vi) G dip{x,y) and (u 2 ,V 2 } G N{{x,y);gph{F)) 
and thus yields V 2 = —vi. Hence {u 2 , —vi) G N((x,y); gph(F)) meaning by definition that 
U 2 G D*F{x,y){vi). Therefore we arrive at 

w = U 1 +U 2 G ui+ D*F{x, y){vi), 

which justifies the inclusion “c” in (9.4) and completes the proof of the theorem. □ 
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Observe that Theorem 9.1 easily implies the chain rule of Theorem 7.2 by setting F(x) := 
{B{x)} and (p{x,y) := f{y) therein. Then we have y{x) = (/ o B){x), 

ri(dom((/?)) = M"’ x ri(dom(/)), ri(gph(T)) = gph(il), 

Thus the relative interiority assumption of Theorem 7.2 yields the validity of the qualihca- 
tion condition (9.5) is satisfied, and we arrive at the chain rule (7.2) directly from (9.4) and 
the coderivative expression in (9.3). 

We now derive from Theorem 9.1 and the intersection rule of Theorem 5.3 a new subdif¬ 
ferential chain rule concerning compositions of convex functions with particular structures. 
We say that g :WP ^ {—oo, oo] is nondecreasing componentwise if 

[xi < yi for all f = 1,... ,p] ^ [g{xi, ...,Xp)< g{yi, ... ,yp)]. 


Theorem 9.2 Define h : M” —> 6?/ h{x) := {fi{x),..., fp{x)), where fi : M"' M 

for i = are convex functions. Suppose that g : MP ^ (— 00 , 00 ] is convex and 

nondecreasing componentwise. Then the composition g o h : MT (— 00 , 00 ] is a convex 
function, and we have the subdifferential chain rule 


p 

d{goh){x) = (71,... ,7p) € 55 ( 2 /), Uj € 5 /i(x), i = 1,... ,p| (9.7) 


i=l 


with X G M” and y := h{x) G dom(5) under the condition that there exist u G M” and 
Ai > fi{u) for all i = 1,... ,p satisfying 


(Ai,..., Ap) G ri(dom(5)). 


Proof. Let F : M” be a set-valued mapping defined by 

F{x) ■= [fi{x),oo) X [/2(x),oo) X ... X [/p(x),oo). 

Then the graph of F is represented by 

gph(F) = {{x,ti, ... ,tp) G M” X MP| ti > fi{x)}. 

Consider further the convex sets 

rij .— {( 33 , Ai,..., Ap) I Aj ^ /j(x)} 

and observe that gph(T) = 0^=1 Since all the functions fi are convex, the set gph(T) 

is convex as well. Define if : M” x ^ (— 00 , 00 ] by ip{x, y) := g{y) and observe, since g is 
increasing componentwise, that 

inf {ip{x,y) I y G F{x)} = g{fi{x),..., fp{x)) = {goh){x), 

which ensures the convexity of the composition g oh] see [14, Proposition 1.54]. It follows 
from Proposition 3.5 and Lemma 5.2 that 

ri(gph(F)) = {(x, Ai,..., Ap) ] A* > ffix) for all f = 1,... ,p}. 
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The assumptions made in the theorem guarantee that ri(gph(T)) n ri(dom((/9)) 7 ^ 0. More¬ 
over, the structure of each set Qi gives us 

[(u,-7i,...,-7p) € N{{x, fi{x),... Jp{x))-,ni)] [(^^,-7i) e N{{x, fi{x));epi{fi)),'yj = 0 if j /i]. 

Using the coderivative dehnition and applying the intersection rule of Theorem 5.3 and also 
Lemma 8.1, we get 


V G i:>*F(x,/i(x),...,/p(x))(7 i,...,7 p) (u,-7 i,...,-7 p) G iV((x, /i(x),...,/p(x)); gph(F))) 


(u, -71,..., -7p) G iV((x, /i(x), ..., fp{x))] Pi fii) 

i=l 

P 

(u,-7i,...,-7p) G ^iV((x,/i(x),...,/p(x));fii) 

i=l 

P 

V = '^Vi with (Ui,-7i) G N{{x, fi{x))-,epifi) 

i=l 

P 

V = '^ Vi with Vi G Jidfiix) 

i=l 

P 

V G ^ 7 i( 9 /i(x). 


2=1 


It follows from Theorem 9.1 that v G d{g o h){x) if and only if there exists a collection 
(71,... ,7p) G dg{y) such that v G D*F{x, fi{x),... ,fp{x)){'yi,... ,7p). This allows us to 
deduce the chain rule (9.7) from the equivalences above. □ 


10 Normals to Preimages of Sets via Set-Valued Mappings 

In this section we calculate the normal cone to convex sets of a special structure that 
frequently appear in variational analysis and optimization. These sets are constructed as 
follows. Given a set 0 C and a set-valued mapping F: M” MP the preimage or inverse 
image of the set 0 under the mapping F is defined by 

F-i(0) := {x G M" I F(x) n0 / 0}. (lO.l) 

0 ur goal here is to calculate the normal cone to the preimage set (10.1) via the normal cone 
to 0 and the coderivative of F. This is done in the following theorem, which is yet another 
consequence of the intersection rule from Theorem 5.3. 

Theorem 10.1 Let F: M” MP be a set-valued mapping with convex graph, and let 0 be 
a convex subset ofMP. Suppose that there exists {a,b) G M"' x MP satisfying 

{a,b) G ri(gph(F)) and b G ri(0). 

Then for any x G F~^{Q) and y G F{x) (iQ we have the representation 

iV(x;F-i( 0 )) = D*F{x,y){N{F,e)). (10.2) 
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Proof. It is not hard to show that F ^(0) is a convex set. Picking any u € iV(x; F ^(0)) 
gives us by definition that 

(u, X — x) < 0 whenever x € F~^{Q), i.e., F{x) n 0 / 0. 

Consider the two convex subsets of defined by 

III := gph(F) and VL 2 := M"' x 0 

for which we have (u,0) G N{{x,y)\VLi D 02 ). Applying now Theorem 5.3 tells us that 
(u, 0) G iV((x, y); Oi) + N{{x, y); fl 2 ) = N{{x, y); gphF) + [{0} x 0)], 
and thus we get the representation 

(u, 0) = (tt, —u) + (0, u) with (u, —u) G A^((x, y); gphF) and uGA^(y;0) 
from which it follows immediately that 

u G D*F{x,y){v) and v G N{y-,Q). 

This versifies the inclusion“c” in (10.2). The opposite inclusion is trivial. □ 


11 Coderivative Calculus 

We see from above that the coderivative notion (9.2) is instrumental to deal with set-valued 
mappings. Although this notion was not properly developed in basic convex analysis, the 
importance of it has been fully revealed in general variational analysis and its applications; 
see, e.g., [4,13,21] and the references therein, where the reader can find, in particular, 
various results on coderivative calculus. Most of these results were obtained in the inclu¬ 
sion form under the corresponding normal qualification conditions generated by (5.8). We 
present below some calculus rules for coderivatives of convex-graph mappings, which are 
derived from the intersection rule of Theorem 5.3 and hold as equalities in a bit different 
form under weaker relative interior qualification conditions. 

Recall that the domain of a set-valued mapping F: M”' is defined by 

dom(F) := {x G M"* I F{x) / 0}. 

Given two set-valued mappings Fi,F 2 : M” ^ M.P, their sum is defined by 

(Fi F 2 ){x) = Fi{x) + F 2 {x) := {yi -k y 2 | yi G Ti(x), y 2 G F 2 {x)}. 

It is easy to see that dom(Fi -|- F 2 ) = dom(Fi) n dom(F 2 ) and that the graph of Fi -|- F 2 is 
convex provided that both Fi , F 2 have this property. 

Our first calculus result concerns representing the coderivative of the sum Fi + F 2 at the 
given point (x,y) G gph(Fi -|- F 2 ). To formulate it, consider the nonempty set 

S{x,y) := {(yi,y 2 ) G x | y = yi -k y 2 , m € Fi{x) for i = 1,2} . 
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Theorem 11.1 Let Fi,F2: M” be set-valued mappings of convex graphs, and let the 

relative interior qualification condition 

ri(gph(Fi)) n ri(gph(F2)) 7^ 0 (11.1) 

hold. Then we have the coderivative sum rule 

D*{Fi + F2){x,y){v)= f| [D*Fi{x,yi){v) + D*F2{x,y2){v)] (11.2) 

(yi,y2)&S{x,y) 

for all [x, y) G gph(Fi + F2) and v G M^. 

Proof. Fix any u G D*{Fi + F2){x,y){v) and (yi,y2) £ S{x,y) for which we have the 
inclusion {u, —v) G A^((3:,y);gph(Ti + F2)). Consider the convex sets 

Qi := {{x,yi,y2) G x x | 2 /i ^ ^^2 := {{x,yuy2) G x x | y2 G F2{x)} 

and deduce from the normal cone definition that 

{u, -V, -v) G N{{x, yi, 1/2); fli n n2)- 

It is easy to observe the relative interior representations 

ri(fli) = {(x,yi,y2) G M” x x | (x,yi) G ri(gph(Fi))}, 
ri(n2) = {(a;,yi,y2) G M" x x | {x,y2) G ri(gph(F2))}, 

which show that condition ( 11 . 1 ) yields ri(fli) nri(fl2) 7^ 0 . It tells us by Theorem 5.3 that 

{u,-v,-v) G N{{x,yi,y2)]^i) + lV((x, yi, ^2); ^^2), 

and thus we arrive at the representation 

{u,-v,-v) = {ui,-v, 0 ) + {u 2 , 0 ,-v) with (uj,-u) G lV((x, y*); gph(Fi)), z = l, 2 . 

This verifies therefore the relationship 

u = ui + U2 £ D*Fi{x,yi){v) + D*F2{x,y2){v), 

which justihes the inclusion “c” in (11.2). The opposite inclusion is obvious. □ 

Next we dehne the composition of two mappings F : M” =f and G : by 

{GoF){x)= U G{y)-.= {z£G{y)\y£F{x)), x G M", 

yeF(x) 

and observe that G o F is convex-graph provided that both F and G have this property. 

Given z £ {G o F){x), we consider the set 

M{x,z) := F{x) n G~^{z). 
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Theorem 11.2 Let F : ML ML and G : M^ M'^ be set-valued mappings of convex 
graphs for which there exist vectors {x,y,z) € M” x x M”? satisfying 

{x,y) € ri(gph(F)) and {y,z) € ri(gph(G)). (11.3) 

Then for any {x,z) € gph(G o F) and w €M^ we have the coderivative chain rule 

D*{GoF){x,z){w)= Pi D*F{x,y)oD*G{y,z){w). (11.4) 

y(iM{x,z) 

Proof. Picking u € D*{G o F){x, z)(w) and y G M{x, z) gives us the inclusion {u, —w) G 
N{{x, ^); gph(G o F)), which means that 

{u, X — x) — {w, z — z) < 0 for all (x, z) G gph(G o F). 

Form now the two convex subsets of M” x x by 

:= gph(F) X and fl2 ■= x gph(G) 

We can easily deduce from the definitions that 

(u, 0, -w) G N((x, y, z); fli D fl 2 ) 

and that the qualification condition (11.3) ensures the validity of the one ri(ni) nri(n 2 ) 7 ^ 0 
in Theorem 5.3. Applying then the intersection rule to the above sets tells us that 

(u, 0, -w) G N((x, y, z); ill n ^ 2 ) = N{{x, y, z); Oi) + N{{x, y, z); O 2 ), 

and thus there is a vector u G such that we have the representation 

(u, 0, -w) = {u, -V, 0) + (0, V, -w) with (u, -u) G N{{x, y); gph(F)), {v,-w) G N{{y, z); gph(G)). 
This shows by the coderivative definition (9.2) that 

u G D*F{x,y){v) and v G D*G{y,z){w), 

and so we justify the inclusion “c” in (11.4). The opposite inclusion is easy to verify. □ 

Our final result in this section provides an exact formula for calculating the coderivative of 
intersections of set-valued mappings , F 2 : ^ defined by 

(Fi n F2){x) := Fi{x) n F2(x), X G M", 

which is also deduced from the basic intersection rule for the normal cone in Theorem 5.3. 

Proposition 11.3 Let Fi,F 2 : M” be of convex graphs, and let 

ri(gph(Fi)) nri(gph(F 2 )) / 0 - 
Then for any y G (Fi D F 2 ){x) and v £MP we have 

D*{FinF 2 ){v)= IJ [D*Fi{x,y){vi) + D*F 2 {x,y){v 2 )]. (11.5) 

Vl+V2=V 
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Proof. It follows from the definition that gph(Fi n F 2 ) = gph(Pi) ngph(P 2 )- Pick any 
vector u G D*{Fi n F 2 ){v) and get by Theorem 5.3 that 

{u, -v) G N{{x, j/); gph(Fi D F 2 )) = N{{x, y); gph(Fi)) + N{{x, y); gph(F 2 )). 

This allows us to represent the pair {u, v) in the form 

{u, -v) = (ui, -Ul) + (U 2 , -U 2 ), 

where (ui,—ui) G A^((x, y); gph(Fi)) and (tt 2 ,—^' 2 ) G i/); gph(T 2 ))- Therefore we have 

u = ui +U 2 ^ D*Fi{x,y){vi) + D*F 2 {x,y){v 2 ) with v = vi + V 2 ., 
verifying the inclusion “c” in (11.5). The opposite inclusion comes from the dehnition. □ 

12 Solution Maps for Parameterized Generalized Equations 

Here we present a rather simple application of coderivative calculus to calculating the 
coderivative of set-valued mappings given in the structural form 

5(x) = {y G RP I 0 G F(x,y)+ G(x,y)}, x G R*", (12.1) 

where F, G : R"’ x RP R"? are set-valued mappings. Mappings of this type can be treated 
as solutions maps to the so-called generalized equations 

0 G F{x, y) + G{x, y), x G R”, y G RP, 

with respect to the decision variable y under parameterization/perturbation by x. This ter¬ 
minology and first developments go back to Robinson [18], who considered the case where 
G{y) = N{y,Q) is the normal cone mapping associated with a convex set 11 and where 
F{x,y) is single-valued. It has been recognized that the generalized equation formalism, 
including its extended form, is a convenient model to investigate various aspects of opti¬ 
mization, equilibrium, stability, etc. In particular, the coderivative of the solution map 
(12.1) plays an important role in such studies; see, e.g., [13] and the references therein. 

To proceed with calculating the coderivative of the solution map (12.1) in the case of 
convex-graph set-valued mappings, we first observe the following fact of its own interest. 

Proposition 12.1 Let F : R"" RP be an arbitrary set-valued mapping with convex graph. 
Given x G dom(F), we have the relationships 

N{x;dom{F)) = D*F{x,y){0) for every y£F{x). (12.2) 

Proof. Picking any v G A^(x;dom(F)) and y G F{x) gives us 

{v,x — x) <0 for all x G dom(F), 
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which immediately implies the inequality 

{v,x - x) + {0,y - y) <0, y e F{x). 

This yields in turn (u,0) € A^((x, y); gph)(T)) and so u G D*F{x,y){0) thus verifying the 
inclusion “c” in (12.2). The opposite inclusion in (12.2) is straightforward. □ 

Theorem 12.2 Let F and G in (12.1) he convex-graph, and let {x,y) € gph5. Impose the 
qualification condition 

ri(gph(F)) n ri(-gph(G)) 7 ^ 0. (12.3) 

Then for every z € F{x,y) fl [—G(3:,y)] we have 

D*S{x,y){v) = IJ {u \ {u,-v) e D*F{{x,y),z){w) + D*G{{x,y),-z){w)}. (12.4) 

weR‘1 

Proof. It is easy to see that the solution map S is convex-graph under this property for F 
and G. Furthermore, we get 

gph(S') = {(x, y) G X RP I 0 G F{x, y) + G{x, y)} 

= {{x,y) G R"' X RP I F{x,y) D [—G{x,y)] 7 ^ 0} = dom(Ff), 

where H{x,y) := F{x,y) D [—G{x,y)]. Take further any u G D*S{x,y){v) and deduce from 
(9.2) and Proposition 12.1 that 

{u,-v) G lV((x,y);gph(S')) = lV((x, y); dom(i?)) = D*H{{x,y,z))fi)) 

for every z € H{x,y) = F{x,y) n [—G{x,y)]. Then the coderivative intersection rule of 
Proposition 11.3 tells us that under the validity of (12.3) that 

(u, -v) G D*H{{x, y, z))(0) = |J [D*F{{x, y), z){w) + D*{-G){{x, y),z){-w)] 

= [J [D*F{{x,y),z){w) + D*{G){{x,y),-z){w)], 

which yields (12.4) and thus completes the proof of the theorem. □ 

Acknowledgement. The authors are grateful to two anonymous referees and the handling 
Editor for their valuable remarks, which allowed us to improve the original presentation. 
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